{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "357ea80e-8ee0-448e-a422-45293f03597c",
   "metadata": {},
   "source": [
    "# Dropout"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcba7a26-afa2-46e4-aa70-d2ae1dd241e2",
   "metadata": {},
   "source": [
    "在2012年，Hinton在其论文《Improving neural networks by preventing co-adaptation of feature detectors》中提出Dropout。当一个复杂的前馈神经网络被训练在小的数据集时，容易造成过拟合。为了防止过拟合，可以通过阻止特征检测器的共同作用来提高神经网络的性能。\n",
    "\n",
    "在2012年，Alex、Hinton在其论文《ImageNet Classification with Deep Convolutional Neural Networks》中用到了Dropout算法，用于防止过拟合。并且，这篇论文提到的AlexNet网络模型引爆了神经网络应用热潮，并赢得了2012年图像识别大赛冠军，使得CNN成为图像分类上的核心算法模型。\n",
    "\n",
    "随后，又有一些关于Dropout的文章《Dropout:A Simple Way to Prevent Neural Networks from Overfitting》、《Improving Neural Networks with Dropout》、《Dropout as data augmentation》。\n",
    "\n",
    "从上面的论文中，我们能感受到Dropout在深度学习中的重要性。那么，到底什么是Dropout呢？\n",
    "\n",
    "Dropout可以作为训练深度神经网络的一种trick供选择。在每个训练批次中，通过忽略一半的特征检测器（让一半的隐层节点值为0），可以明显地减少过拟合现象。这种方式可以减少特征检测器（隐层节点）间的相互作用，检测器相互作用是指某些检测器依赖其他检测器才能发挥作用。\n",
    "\n",
    "Dropout说的简单一点就是：我们在前向传播的时候，让某个神经元的激活值以一定的概率p停止工作，这样可以使模型泛化性更强，因为它不会太依赖某些局部的特征."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff8486f9-f4ce-4c5b-b330-967e946bfb48",
   "metadata": {},
   "source": [
    "本小节主要介绍其在mindspore中的用法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "09d89fda-298a-4f72-8713-398eca09dc03",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import nn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fc9c066-b95c-402b-b131-7fd44335b4b1",
   "metadata": {},
   "source": [
    "调用mindspore中的dropout函数，然后训练一个现有网络，得到一个结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e50b32d0-68b6-49a9-9927-33bd7ca8d833",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training result:\n",
      " [[[0.        1.4285715 1.4285715]\n",
      "  [0.        1.4285715 1.4285715]]\n",
      "\n",
      " [[1.4285715 1.4285715 1.4285715]\n",
      "  [1.4285715 0.        1.4285715]]]\n",
      "\n",
      "infer result:\n",
      " [[[1. 1. 1.]\n",
      "  [1. 1. 1.]]\n",
      "\n",
      " [[1. 1. 1.]\n",
      "  [1. 1. 1.]]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import mindspore as ms\n",
    "\n",
    "x = ms.Tensor(np.ones([2, 2, 3]), ms.float32)\n",
    "net = nn.Dropout(keep_prob=0.7)\n",
    "\n",
    "# 执行训练\n",
    "net.set_train()\n",
    "output = net(x)\n",
    "print(\"training result:\\n\", output)\n",
    "\n",
    "# 执行推理\n",
    "net.set_train(mode=False)\n",
    "output = net(x)\n",
    "print(\"\\ninfer result:\\n\", output)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
